1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)? The flaw that we found was the use of the word "they" in the sentence.
Evidence: Commandment 2: Commandment 2: They refers to the district administrators but to students they may be confused and think that "they" refers to the teacher. They use of two concepts in this sentence i.e. district administrators and teachers is too complex for students. Thus there may be confusion as to who "they" is referring to.
Binary Choice Item # 2 - flaw(s)?4. Thou shall not employ complex syntax in assessment items. * Avoid complicated sentence construction – use very simple sentences.
Evidence: The choice of vocabulary words are complex for the students to comprehend. There are multiple complex vocabulary words in this sentence thus it may come off to be confusing for the students. The sentence is also long and because there are so many complex words the students will have a hard time pronouncing them and understanding what they mean. The majority of time will be used in looking up the words to answer the question itself.
Binary Choice Item # 3 - flaw(s)?The statement “it is almost impossible” is the flaw.
Evidence: Commandment 3: Thou shall not provide students with unintended clues regarding appropriate responses. Absolute qualifiers such as never and always often appear in true-false tests. Using “it is almost impossible” makes #3 a trick question. The focus on good assessment should be on finding out what students know, not to trick them.
Binary Choice Item # 4 - flaw(s)?The flaw is the consistent use of absolute qualifiers and complex syntax.
Evidence:Commandment 4: Thou shall not employ complex syntax in assessment items. They should avoid complex sentence structure and instead, use simple sentences. Commandment 3 states that one should avoid the use of absolute qualifiers, in this case, the binary choice contained the absolute qualifier “never”. It is also confusing, and should be restated in a more simple manner in order for understanding.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
Heather is ten years older than Joel, and next year she will be twice as old as Joel.
True or False?
Joel is younger than Heather.
Next year Joel will be 10.
This year Heather is 12.
This year Joel is 9.
Answer Key: Number 1: True Number 2: True Number 3: False Number 4: True
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)?Short stem & long alternative – does not mesh well. Also, the use of the word “not”
Evidence:Lengthy stems and terse alternatives are preferred over skimpy stems and long alternatives. The stem is way too short in comparison to the alternative. A well constructed multiple-choice stem presents a clearly described task so that the student can get to work. Since the stem and alternative do not mesh well, the choice item gives the student no room for inferences and generalizations. One should also avoid using negatively stated stems because it can be confusing for students.
Multiple-Choice Item # 2 - flaw(s)?The use of the word “an”.
Evidence:Unintended clues occur when test writers provide an incomplete statement, as stated in Commandment 3. The unintended clue in this case is the word “an” giving the test-taker a clue that the answer is going to begin with a vowel. The multiple choice item gives only one choice with a vowel, so even if the student did not know the answer, they will figure it out by finding the answer that contains the vowel at the beginning, and in this case there was only one.
Multiple-Choice Item # 3 - flaw(s)?The flaw is the option to use ALL THE ABOVE! The power point states that students who do not know the answer may infer that if the teacher went into great detail for one answer and there must be something ‘special’ about that choice option. In addition, the student will second guess their answers thinking that the use of ALL THE ABOVE was intended to be there for a reason and will usually choose that option.
Evidence:Commandment number 3 Thou shall not provide students with unintended clues regarding appropriate responses. The use of ALL THE ABOVE can trick student. The power point states, the focus of good assessment should be on finding out what students know and do not know . . . not on trick questions.
Multiple-Choice Item # 4 - flaw(s)?Too much attention to answer choice D.
Evidence:Slide 40 clearly states that item writers frequently fail to realize that the length of a multiple-choice item can give an unintended clue to the correct answer. Students can figure that if the teacher has given so much attention to constructing an answer options there must be something special about it proving that D is correct.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.” On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Column A
Column B
1. Bias-Detection
A. Determining DIF indices
2. Item Difficulty
B. Computing K-R formula
3. Reliability
C. Evaluating curricular alignment
4. Score Interpretation
D. Deriving percentiles
5. Validity
E. Calculating p// - values
Matching Item # 1 - flaw(s)?Each column is equal.
Evidence:Slide 51 clearly states that one should make sure there are at least a few extra responses – otherwise a student who knows 80% of the matches to the premises may be able to figure out the remaining matches - guessing and/or process of elimination. Either way – this does not give you accurate data about the student. This matching has 5 concepts in Column A to be matched with 5 procedures in Column B, so the student can use process of elimination to get all correct without even knowing all of the information.
1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)? The flaw that we found was the use of the word "they" in the sentence.
Evidence: Commandment 2: Commandment 2: They refers to the district administrators but to students they may be confused and think that "they" refers to the teacher. They use of two concepts in this sentence i.e. district administrators and teachers is too complex for students. Thus there may be confusion as to who "they" is referring to.
Binary Choice Item # 2 - flaw(s)? 4. Thou shall not employ complex syntax in assessment items.
* Avoid complicated sentence construction – use very simple sentences.
Evidence: The choice of vocabulary words are complex for the students to comprehend. There are multiple complex vocabulary words in this sentence thus it may come off to be confusing for the students. The sentence is also long and because there are so many complex words the students will have a hard time pronouncing them and understanding what they mean. The majority of time will be used in looking up the words to answer the question itself.
Binary Choice Item # 3 - flaw(s)? The statement “it is almost impossible” is the flaw.
Evidence: Commandment 3: Thou shall not provide students with unintended clues regarding appropriate responses. Absolute qualifiers such as never and always often appear in true-false tests. Using “it is almost impossible” makes #3 a trick question. The focus on good assessment should be on finding out what students know, not to trick them.
Binary Choice Item # 4 - flaw(s)? The flaw is the consistent use of absolute qualifiers and complex syntax.
Evidence: Commandment 4: Thou shall not employ complex syntax in assessment items. They should avoid complex sentence structure and instead, use simple sentences. Commandment 3 states that one should avoid the use of absolute qualifiers, in this case, the binary choice contained the absolute qualifier “never”. It is also confusing, and should be restated in a more simple manner in order for understanding.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
Heather is ten years older than Joel, and next year she will be twice as old as Joel.
True or False?
- Joel is younger than Heather.
- Next year Joel will be 10.
- This year Heather is 12.
- This year Joel is 9.
Answer Key:Number 1: True
Number 2: True
Number 3: False
Number 4: True
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)? Short stem & long alternative – does not mesh well. Also, the use of the word “not”
Evidence: Lengthy stems and terse alternatives are preferred over skimpy stems and long alternatives. The stem is way too short in comparison to the alternative. A well constructed multiple-choice stem presents a clearly described task so that the student can get to work. Since the stem and alternative do not mesh well, the choice item gives the student no room for inferences and generalizations. One should also avoid using negatively stated stems because it can be confusing for students.
Multiple-Choice Item # 2 - flaw(s)? The use of the word “an”.
Evidence:Unintended clues occur when test writers provide an incomplete statement, as stated in Commandment 3. The unintended clue in this case is the word “an” giving the test-taker a clue that the answer is going to begin with a vowel. The multiple choice item gives only one choice with a vowel, so even if the student did not know the answer, they will figure it out by finding the answer that contains the vowel at the beginning, and in this case there was only one.
Multiple-Choice Item # 3 - flaw(s)?The flaw is the option to use ALL THE ABOVE! The power point states that students who do not know the answer may infer that if the teacher went into great detail for one answer and there must be something ‘special’ about that choice option. In addition, the student will second guess their answers thinking that the use of ALL THE ABOVE was intended to be there for a reason and will usually choose that option.
Evidence: Commandment number 3 Thou shall not provide students with unintended clues regarding appropriate responses. The use of ALL THE ABOVE can trick student. The power point states, the focus of good assessment should be on finding out what students know and do not know . . . not on trick questions.
Multiple-Choice Item # 4 - flaw(s)? Too much attention to answer choice D.
Evidence:Slide 40 clearly states that item writers frequently fail to realize that the length of a multiple-choice item can give an unintended clue to the correct answer. Students can figure that if the teacher has given so much attention to constructing an answer options there must be something special about it proving that D is correct.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”
On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Matching Item # 1 - flaw(s)?Each column is equal.
Evidence:Slide 51 clearly states that one should make sure there are at least a few extra responses – otherwise a student who knows 80% of the matches to the premises may be able to figure out the remaining matches - guessing and/or process of elimination. Either way – this does not give you accurate data about the student. This matching has 5 concepts in Column A to be matched with 5 procedures in Column B, so the student can use process of elimination to get all correct without even knowing all of the information.